The JPEG (Joint Photographic Experts Group) and MPEG (Moving Picture Experts Group) standards for image compression have been adopted for many diverse digital imaging applications. A commonality between these standards is the use of the discrete cosine transform (DCT). In both JPEG and MPEG, the pixel values that represent an original image are divided into non-overlapping, contiguous 8×8 blocks. Each 8×8 block is then transformed with the DCT to produce 64 spatial frequency coefficients. The lowest frequency coefficient is termed the DC coefficient, and it is related to the mean of the original 8×8 block of pixel values. The other 63 coefficients are termed the AC coefficients, and they represent the fluctuations of the pixel values around the mean. The 8×8 DCT that is used in JPEG and MPEG is an example of a block-transform.
During the compression process, the frequency coefficients for each 8×8 block are quantized to reduce the number of possible values. The degree of quantization for each coefficient is specified by a quantization table (or q-table) that consists of 64 values, one for the DC coefficient and one for each of the 63 AC coefficients. A larger q-table value for a given coefficient indicates more quantization. The quantized coefficients are then converted to a binary representation using various encoding methods. The binary representation is the data that is stored or transmitted in a compressed file. To reconstruct an image from the compressed file, the binary representation is decoded to recover the quantized coefficients for each 8×8 block. The quantized coefficients are then dequantized to produce dequantized coefficients, which are an approximation to the original frequency coefficients. Finally, the dequantized coefficients are transformed into an 8×8 block of reconstructed pixel values using an inverse DCT.
An image that is reconstructed from a compressed file consists of the concatenation of 8×8 blocks of pixel values that have been quantized in the DCT frequency domain. The quantization of the coefficients is the mechanism that allows a tradeoff between amount of compressed data and the quality of the reconstructed image. It is well known that an image that has been compressed using the DCT can produce objectionable artifacts at high compression levels, which corresponds to high levels of quantization. These artifacts include blocking artifacts, which are primarily the result of coarse quantization of the low frequency coefficients (particularly the DC coefficient), and ringing artifacts, which are primarily the result of coarse quantization of the mid/high frequency AC coefficients. Blocking artifacts are produced largely by discrepancies between the mean values of adjacent 8×8 blocks, and these discrepancies are particularly noticeable in image regions that have low detail, i.e., regions that have slowly varying content such as sky or water. Ringing artifacts occur near edges, i.e., abrupt changes from light to dark or vice versa, and they are particularly noticeable when there is a low detail region on one or both sides of the edge. As used herein, regions that are adjacent to low detail regions are termed boundary regions.
Numerous techniques have been described in the patent and technical literature to reduce the blocking and ringing artifacts that occur at high compression levels in compressed images. Some techniques are aimed at removing only the blocking artifacts (usually termed “deblocking” techniques), while other techniques attempt to address both types of artifacts, often through the use of different processing methods for the blocking artifacts and the ringing artifacts. A common problem for all of these techniques is providing an adequate reduction in the artifacts while still preserving desirable image content.
In our experience, one of the best methods for artifact reduction is that described in commonly-assigned U.S. Pat. No. 6,427,031, “Method for removing artifacts in an electronic image decoded from a block-transform coded representation of an image,” to J. R. Price. This patent is incorporated herein in its entirety by reference. Price teaches a method for artifact reduction that segments an image into low detail regions and boundary regions, and then applies different processing steps to each region. While this method provides a significant reduction in the appearance of artifacts, particularly at high compression levels, it still suffers from several disadvantages. First, the type of processing that is applied to the boundary regions can result in a blurring of desirable image content in these regions, leading to a loss in apparent sharpness. Second, the method does not adapt to the compression level, so the same processing is applied regardless of the severity of the artifacts. This lack of adaptivity is common to most artifact reduction methods, and it can lead to additional blurring of desirable image content. Third, the processing parameters for the low detail and boundary regions are chosen independently of one another, which can lead to noticeable switching artifacts at the interface of the low detail and boundary regions. Finally, the high detail regions, i.e., regions that are not low detail or boundary, are not processed even though artifacts may be present. In the high detail regions, the presence of image content that has significant activity provides a masking signal that renders the artifacts less obvious. However, it may be desirable to provide some artifact reduction in the high detail regions as well.
Thus, there is a need for an artifact reduction method that does not blur desirable image content while still providing a significant reduction in the artifacts. Moreover, the method should automatically adapt to the compression level so that the degree of artifact reduction is matched to the severity of the artifacts. Finally, the method should provide a unified framework so that the different regions (low detail, boundary, and high detail) are processed in such a way that the transitions between the regions are not objectionable in the final image.